Capacity Verification for High Speed Network Intrusion Detection Systems

Commercially available network intrusion detection systems (NIDS) came onto the market over six years ago. These systems have gained acceptance as a viable means of monitoring the security of consumer networks, yet no commercial standards exist to help consumers understand the capacity characteristics of these devices. Existing NIDS tests are flawed. These tests resemble the same tests used with other networking equipment, such as switches and routers. However, switches and routers do not conduct the same level of deep packet inspection, nor do they require the higher-level protocol awareness that a NIDS demands. Therefore, the current testing does not allow consumers to infer any expected performance in their environments. Designing a new set of tests that specific to the weak areas, or bottlenecks, of a NIDS is the key to discovering metrics meaningful to the consumers. Consumers of NIDS technology can then examine the metrics used in the tests and profile their network traffic based on these same metrics. Consumers can use standard test results to accurately predict performance on their networks. This paper proposes a test methodology for standardized capacity benchmarking of NIDS. The test methodology starts with examination of the bottlenecks in a NIDS, then maps these bottlenecks to metrics that can be tested, and finally explores some results from tests conducted. Introduction and Scope Currently, no industry standards exist for testing any aspect of network intrusion detection systems (NIDS). The NIDS industry is maturing along the same lines as the router, switch, and firewall industries that came before it, and has now reached the point where standardization of testing and benchmarking is possible. Attempting to define a testing standard is beyond the scope of this paper. Instead, the metrics and methodology used to properly verify the capacity of high-speed NIDS are explored. Performance of NIDS is usually defined by false positive and false negative ratios, and speed or capacity. This paper addresses the issue of benchmarking the capacity of a NIDS. This paper uses capacity to refer to the ability of a NIDS to capture, process, and perform at the same level of accuracy under a given network load as it does on a quiescent network. Gauging the capacity of a NIDS is difficult. Several variables in the characteristics of the network traffic affect the performance of a NIDS. The last year has seen claims of NIDS performing at or near gigabit speeds. In every case, however, further investigation by reasonably sophisticated NIDS practitioners revealed critical flaws in the testing methodology. The variety of technology used to perform network-based intrusion detection further complicates finding the proper metrics. The following technologies are used for NIDS: • Stateless inspection of the packets or packet headers • Protocol decode and analysis • Regular expression matching of packet data • Anomaly detection